The Life of a Message
Our digital lives are so immersed in complex technologies that we take them for granted, fail to understand how they really work or are sometimes even oblivious to their existence. An example of such a technology is the one behind messaging apps which are used literally by every person on this planet with a phone.
Let’s look at the technology powering Status messenger. In previous articles, we looked at how our messenger broadly works and compared it with other popular messengers. In this article, we trace the “life of a message” when Alice communicates with Bob using Status messenger. In the context of enabling this message’s journey, we describe in detail the various components of the underlying peer-to-peer (P2P) network, privacy protocols, cryptographic algorithms and other messaging primitives used to architect the security and privacy properties of Status messenger.
In a client-server paradigm — as used in popular messengers including Signal and Telegram — many of the above aspects are relatively straightforward because clients talk to servers which are generally trusted, available and aware of how to route messages to other clients. Status messenger is however architected for decentralized P2P networks which don’t have the concept of servers and clients.
Decentralized P2P messaging aims for removal of centralized rent-seeking intermediaries, removal of single points of failure and increased resistance to censorship.
Every node is a peer, sometimes with different capabilities. Messages don’t have only two hops — from the source client to the server and then to the destination client — as is the case in client-server architectures. They instead hop across multiple peers and continue hopping even after reaching their intended recipient because peers do not know who is the intended recipient.
We aim to provide privacy at the transport layer where nodes can achieve plausible deniability of having received messages meant for them. We also use cryptographic keys as account identifiers instead of the commonly used phone numbers for reasons of enhanced privacy and pseudonymity (as described in this article). These aspects naturally add more complexity to the architecture of our solution.
Simply put, the journey of a message from Alice to Bob starts with Alice knowing Bob’s Status chat key and then flooding the P2P network with her encrypted message. This is relayed across peers and eventually reaches Bob who is the only one that can decrypt this message from Alice. Alice and Bob also know how to “listen” for each other’s messages without anyone else on the network inferring them to be the intended recipients of messages.
This, in short, is the private life of a message on Status messenger.
Admittedly, our current implementation is neither fully P2P nor completely decentralized because it makes certain simplifying practical assumptions, for now, to deliver a working messenger product. For e.g., parts of our current implementation resemble a client-server relationship and Status runs all the message relaying/storing nodes today. Nevertheless, it is possible for anyone to run any kind of node in our architecture — specifications are public, all the code is open-source and network participation is permissionless. What is missing today is the lack of incentives to run peer nodes for relaying and (temporarily) storing messages for others and this is an active area of research at Status.
This article therefore focuses on describing our envisioned architecture, towards which we are steadfastly headed, instead of the current implementation which is constantly evolving towards that vision.
We describe the architectural building blocks of Status messenger and illustrate the journey of Alice’s message to Bob through these different layers. This article attempts to give deep technical insights into our protocol stack by consolidating key aspects from across our various specifications on client, account, secure transport, payloads, Waku usage and MVDS, while also borrowing from Signal’s specifications on X3DH and Double Ratchet.
The illustration above shows the five layers of Status messenger protocol stack with their respective purposes in the middle and the specific technologies which constitute that layer to the right.
The bottommost layer is a P2P overlay on the TCP/IP network. This layer implements a public, permissionless peer-to-peer network, powered by the devp2p network protocols.
Devp2p provides protocols for node discovery where peers use TCP-based RLPx transport protocol to communicate with each other. Status’s Vac team is currently working on a libp2p replacement for devp2p which will support multiple transports, better protocol negotiation and NAT traversal among other benefits.
Node Types: We define a node by its set of capabilities. These capabilities are: send messages, receive messages, relay messages, store historical messages and bootstrap other nodes.
Based on these capabilities, we define the following types of nodes:
- 1Light node: only has send and receive capability. These are typically mobile clients with Status messenger that are resource-constrained and so can only send and receive their own messages.
- 2Full node: has all capabilities of send/receive/relay/store/bootstrap. These nodes send, receive and relay messages for other nodes in the network. They may temporarily store messages meant for other nodes that are currently offline. They may also help bootstrap nodes by helping them discover and connect to other nodes in the network.
- 3Bootstrap node: only has capability to bootstrap nodes. These nodes only help bootstrap nodes by helping them discover and connect to other nodes in the network.
Note that these capabilities are configurable and can change over time based on node capacity or context. For e.g., a mobile client might behave like a Light node on cellular networks but switch to being a Full node on Wifi.
Node Discovery: A Status node must discover other peers or have a list of peers to connect to. Status supports a combination of Discovery v5 and Rendezvous protocols with an option to also use static nodes.
Discovery v5 uses bootstrap nodes to discover other peers. It is a Kademlia-based discovery mechanism and might consume a significant (at least on mobile) amount of network traffic to operate. (See this article for more details.)
Rendezvous protocol is a simpler and more mobile-friendly peer discovery mechanism. It is a request-response discovery mechanism and uses Ethereum Node Records (ENR) to report discovered peers.
Alternatively, a client may rely exclusively on a list of static peers. While this may be the most efficient way because there is really no discovery involved, the disadvantage is that these peers might be offline or unavailable, and without a peer discovery mechanism, it won’t be possible to find new ones.
The transport privacy layer provides routing, metadata protection, topic-based multicasting and encryption algorithms to support asynchronous chat. This is implemented by Waku, which is our successor to Whisper, as explained in this article.
Waku uses the concept of topics to partition its messages. Topics are strings derived using specified algorithms and are used in Waku “envelopes”, where envelopes encapsulate encrypted messages along with this topic and time-to-live (TTL).
Given that in a P2P network, all messages are broadcast to all nodes, nodes have to “listen” to certain topics of their interest. While in theory everyone could communicate using a single topic, that would be extremely inefficient because nodes will have to try to decrypt every message — since anyone can receive Waku envelopes, it relies on the ability to decrypt messages to decide the correct recipient. The other end of the spectrum would use a unique topic for each conversation but that does not provide the required privacy because it would be much easier to detect whether and when two parties have an active conversation.
Status messenger strikes a balance to provide the desired level of transport privacy by using three types of topics:
- 1Contact code topic: Given that Status messenger uses cryptographic keys as account identifiers and that there are no central servers in the P2P network, nodes need a way to bootstrap the cryptographic material required to initiate a conversation with other nodes. Contact code topic facilitates this bootstrapping.
Contact code topic is defined as "0x” + publicKey + "-contact-code", where publicKey is a node’s public chat or identity key. The public chat key is derived from the user’s BIP-39 seed phrase on m/43’/60’/1581’/0/0 path as per EIP-1581. Every node broadcasts an initial cryptographic bundle (described in next section) on this topic.
So if Alice is interested in talking to Bob, she listens on this topic with Bob’s public chat key (in the equation above) which we assume she already knows. When Bob’s broadcasted message (nodes keep publishing this periodically) on this topic reaches Alice, she has Bob’s initial cryptographic bundle required to initiate a chat with him. - 2Partitioned topic: Alice has to now communicate with Bob to generate specific cryptographic keys which will be used to encrypt future messages between them. Partitioned topic is used for this communication.
Partitioned topic is defined as "contact-discovery-" + mod(publicKey, 5000), where it is derived from the node’s public chat key split into 5000 partitions. The number of partitions affects the trade-off between efficiency and privacy — more partitions provide lesser privacy with greater efficiency, and vice versa. All nodes keep listening on their respective partitioned topics to check if any other nodes are interested in communicating with them.
Alice initiates communication with Bob on his partitioned topic which is used until Alice and Bob generate a shared symmetric key between them. - 3Negotiated topic: Once a symmetric cryptographic key is generated between Alice and Bob, they must start communicating using a topic specific to this 1:1 chat. Negotiated topic is used for this communication thereafter.
Negotiated topic is defined as "0x" + first four bytes of keccak256(symmetricKey) + “-negotiated", where symmetricKey is the symmetric cryptographic key generated between Alice and Bob using an extended version of the Diffie-Hellman algorithm.
So, to summarize, the topics used in the message flow between Alice and Bob are as follows:
- 1Alice listens to Bob’s contact code topic to get his cryptographic bundle.
- 2Alice sends a message on Bob’s partitioned topic. Alice and Bob generate a shared symmetric key using an extended version of Diffie-Hellman algorithm (see next section) using the partitioned topics.
- 3Alice and Bob start using the negotiated topic derived from the shared symmetric key.
If Alice does not receive Bob's cryptographic bundle while listening for Bob's contact code topic (in step 1 above), she sends an encrypted message with her cryptographic bundle to Bob on his partitioned topic, upon which Bob engages with the protocol from step 2 onwards.
For public group chats, the topic is the first four bytes of the hash of the channel name. Private group chats do not have a dedicated topic — messages are sent 1:1 to multiple recipients using the contact code, partitioned and negotiated topics similar to 1:1 private messages.
Waku also adds a layer of encryption by encrypting messages with the receiver’s identity key (in case of 1:1 messages or private group chats) or with a symmetric key in case of public messages.
This layer is responsible for providing cryptographic guarantees of confidentiality, integrity, authenticity and forward secrecy. It is transport-agnostic and works over asynchronous networks i.e. doesn’t require both communicating entities to be online at the same time. It is based on X3DH and Double Ratchet specifications from Signal, with some adaptations to operate in a P2P environment.
X3DH: X3DH, which stands for Extended Triple Diffie-Hellman, is used to establish a shared secret key between two parties who mutually authenticate based on their public keys. It provides forward secrecy and plausible deniability in asynchronous settings.
X3DH uses the concept of identity keys, ephemeral keys and signed prekeys during the protocol run which ends in Alice (A) and Bob (B) together generating a shared symmetric key. The three Diffie-Hellman rounds involved are:
1. DH-1 = DH(IK-A, SPK-B)
2. DH-2 = DH(EK-A, IK-B) and
3. DH-3 = DH(EK-A, SPK-B)
and the final shared symmetric key (SK) is calculated as:
SK = KDF(DH-1 || DH-2 || DH-3)
where IK = Identity Key, EK = Ephemeral Key, SPK = Signed PreKey, DH = Diffie-Hellman and KDF = Key Derivation Function.
The X3DH protocol specified by Signal involves three parties: Alice, Bob and a server. But in our P2P setting, where there is no server, the X3DH cryptographic bundles which contain IK (public chat key in our case), SPK and PreKey signatures are exchanged over Waku contact code topic as described earlier.
Not relying on servers to distribute X3DH bundles makes it harder for an external observer to gather metadata and thus improves privacy. Additionally, our implementation does not use One-time PreKeys (OPK as described in the Signal spec) because we can not ensure that they are used only once without an interactive protocol or storing them on servers which don’t exist in our P2P setting. We instead rotate keys more frequently.
The X3DH protocol phases are run using Waku’s partitioned topic as described earlier to generate a shared symmetric key between Alice and Bob. This shared symmetric key is then used to seed the derivation of encryption keys for future messages using the Double Ratchet algorithm and also in deriving Waku’s negotiated topic used thereafter.
Double Ratchet: Double Ratchet algorithm is used by Alice and Bob to exchange encrypted messages based on the shared secret key (generated by X3DH) which provides end-to-end encryption (E2EE) and perfect forward secrecy (PFS).
New keys are derived for every message so that earlier keys cannot be calculated from later ones (forward secrecy). Messages also contain new Diffie-Hellman public keys which are mixed into the derived keys so that later keys cannot be calculated from earlier ones. These techniques protect encrypted messages from the past or in the future if current keys are compromised.
Double Ratchet algorithm uses the concept of KDF chains. In a session between Alice and Bob, each entity stores a KDF key for three chains: a root chain, a sending chain and a receiving chain. Alice's sending chain matches Bob's receiving chain, and vice versa.
The Diffie-Hellman output secrets become the inputs to the root chain and the output keys from the root chain become new KDF keys for the sending and receiving chains. This is called the Diffie-Hellman ratchet. The sending and receiving chains advance as each message is sent and received and their output keys are used to encrypt and decrypt messages. This is called the Symmetric-key ratchet. The combination of Diffie-Hellman and Symmetric-key ratchets is known as the Double Ratchet algorithm which is shown in the above illustration from Signal’s specification. (Watch this video for a detailed explanation of this protocol.)
This layer implements the Minimum Viable Data Synchronization (MVDS) protocol which ensures data consistency with reliable messaging between peers across an unreliable P2P network where peers may be unreachable or unresponsive.
The payloads at this layer consist of records which are of four types: Offers, Requests, Messages and Acks. When Alice has a message to send to Bob, she sends an Offer. If Bob responds with a Request, she further sends him the Message to which he responds back with an Ack. A maximum of one payload should be sent per epoch. Messages may be sent either in Interactive or Batch mode.
The above diagram illustrates the flow of payloads in interactive mode. Send counts and epochs are tracked for retransmission of payloads.
This layer is responsible for implementing the end user functionality of 1:1 private messages, private group chats and public chats. The below illustration shows the different parts of a message as described in the payload specification.
The message type (1:1, private group, public group) determines how to encrypt a particular message and what metadata needs to be attached. The content type determines the actual payload of the message. For e.g., the content could be plain text, sticker, emoji, image or an audio message. All payloads are encoded using Protobuf.
Now that we have described all the layers of our protocol stack and even traced Alice’s message through parts of it, let’s summarize the entire sequence of steps for Alice to communicate with Bob using Status messenger. We will assume that both Alice and Bob have Status messenger app installed on their respective mobile devices which represent Status Light nodes.
- 1Peer Discovery: The Status messenger apps on Alice and Bob’s mobile devices discover Status peers in the P2P network that are Full nodes using the Bootstrap nodes. These Full nodes are responsible for relaying Alice’s messages to Bob or even storing them temporarily if Bob is not online.
- 2Contact Discovery: The first step is for Alice to discover Bob on the Status messenger. This can happen in multiple ways such as scanning Bob’s Status QR code, adding Bob’s chat key out of band, searching Bob’s ENS name or adding Bob’s chat key via a shared public channel.
- 3Contact Verification: Alice can then optionally verify that Bob’s public key really belongs to who she thinks is Bob by checking the corresponding three-word pseudonym, Bob’s ENS name, or doing some out of band verification.
- 4Initial Key Exchange: Alice then gets Bob’s X3DH bundle by listening on Bob’s contact code topic as described earlier in the section on Waku.
- 5Secure Channel Creation: X3DH and Double Ratchet protocols run over Waku partitioned and negotiated topics respectively to help create a secure end-to-end encrypted channel between Alice and Bob.
- 6Message Transfer: Having established a secure channel, Alice can now send 1:1 private messages to Bob. The encrypted messages get wrapped in Waku envelopes with the negotiated topic and a TTL (15 seconds currently). Messages propagate among Status peer nodes for a little while, until their TTL expires, and then get saved in Status Full nodes for up to 30 days. If Bob is online at that time, he receives Alice’s message. If not, he will have to retrieve it from the Full nodes at a later time.
- 7Message Notification: With notifications recently released in version 1.4 on Android, Bob will also receive a notification if he is on Android. Notifications on iOS is a work in progress.
- 8Message Retention: Once Bob’s app receives the message, it gets stored in the app’s chat database encrypted with a key derived from the user’s BIP-39 seed phrase on m/43’/60’/1581’/1/0 path as per EIP-1581.
We have described the five layers of the Status protocol stack and illustrated the journey of Alice’s message to Bob through these different layers.
The two bottommost layers that implement P2P overlay with the different node types and transport privacy with Waku topics significantly deviate from the standard client-server paradigm architectures used by other messengers. The protocols used in the Secure Transport layer that provide cryptographic guarantees have been adapted for our decentralized P2P network. The two topmost layers that implement data synchronization and manage payloads for the three different chat modes are unique to Status messenger’s protocol stack.
This article hopefully gives you key technical insights into the journey of Alice’s message to Bob as it navigates through these protocol layers and hops across multiple nodes with a single purpose in its life — privacy.
Status messenger aims to be the ultimate privacy-preserving messenger built using decentralized P2P technology.
(Thanks to Andrea Piana, Corey Petty, Jonathan Zerah and Tobias Heldt for reviewing drafts of this article and providing helpful feedback. Thanks to Alex Howell for the thoughtful illustrations. And, thanks to authors of specifications for providing the foundational material for this article.)
Use a private messenger now and install Status here in the App Store, Playstore, and via APK.
Learn more about all the features of Status including the private messenger, secure crypto wallet, and web3 browser here.
